Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow writing JSON reporter to write directly to file #3255

Closed
wants to merge 6 commits into from

Conversation

mkamioner
Copy link

Description of the Change

Add feature for reporterOption in JSON reporter to output directly to file system. Method is similar to how xunit is implemented

Alternate Designs

It would be simple to write the JSON to the stdout and then just pipe that into a file. However when using certain tools like mocha-multi-reporters there is no way around writing JSON and the spec reporter to the stdout. Furthermore, with bash and jq JSON is a great way to parse the results while still allowing for human readable output with spec.

Why should this be in core?

There are so many different JSON reporters but non are managed and they all fall through. This is a simple fix that would be a great feature that will help many

Benefits

The ability to write JSON files of your output thereby making analyzation of your tests easier to parse

Possible Drawbacks

None that I can think of right now (happy to hear if someone has some ideas)

Applicable issues

This is an enhancement (minor release) I think

@jsf-clabot
Copy link

jsf-clabot commented Feb 28, 2018

CLA assistant check
Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Mo Kamioner seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.

@coveralls
Copy link

coveralls commented Feb 28, 2018

Coverage Status

Coverage increased (+0.02%) to 90.009% when pulling 55b1f82 on mkamioner:master into f71f347 on mochajs:master.

@boneskull
Copy link
Contributor

Thanks. I'm not convinced Mocha needs this feature, since you can pipe to a file, like you mentioned.

If we do want to be able to write directly to a file (and I'm not sure how necessary that is for the userbase, because pipes), I'd rather make it happen in the general case by swapping the output stream, instead of a reporter-by-reporter basis.

Please don't let this discourage you from contributing to Mocha! There are plenty of issues labeled help wanted that we could use help with.

@boneskull boneskull closed this Mar 1, 2018
@plroebuck
Copy link
Contributor

Guess I wasted my time doing basically the same enhancement, except I threw out the existing "json.spec.js" and based my tests on revamped version of XUnit's spec (which did stream checking). I have a use case for our project (which programmatically uses Mocha under the covers for RAML-based test automation) referenced in this issue (for one). If someone adds console.log() statements to his hook files, but specifies (for example) the Mocha json reporter -- assertions will fail due to crossing of the streams and Abao can't salvage Mocha's output. I was actually rather stunned that file output was ONLY available for XUnit reporter -- IMO, output should be trivially redirectable (-o | --output) without having to convert individual reporters as such or involve command shell piping.

@plroebuck
Copy link
Contributor

@boneskull BTW, had two questions left after doing my enhancement -- could you shed some light on them?

  1. These two lines. First binds and assigns to a variable. But why is the expectation seemingly referring to result of its invocation.

    var boundXUnit = XUnit.bind({}, runner, options);
    expect(boundXUnit).to.throwException('file output not supported in browser');

  2. Reports fake times when used with sinon.useFakeTimers(). #237

    /**
    * Save timer references to avoid Sinon interfering (see GH-237).
    */
    /* eslint-disable no-unused-vars, no-native-reassign */
    var Date = global.Date;
    var setTimeout = global.setTimeout;
    var setInterval = global.setInterval;
    var clearTimeout = global.clearTimeout;
    var clearInterval = global.clearInterval;
    /* eslint-enable no-unused-vars, no-native-reassign */

    /**
    * Save timer references to avoid Sinon interfering.
    * See: https://github.com/mochajs/mocha/issues/237
    */
    /* eslint-disable no-unused-vars, no-native-reassign */
    var Date = global.Date;
    var setTimeout = global.setTimeout;
    var setInterval = global.setInterval;
    var clearTimeout = global.clearTimeout;
    var clearInterval = global.clearInterval;
    /* eslint-enable no-unused-vars, no-native-reassign */

If these variables were exported from Base, would the individual reporters need to have same code repeated?

@boneskull
Copy link
Contributor

  1. XUnit.bind() returns a function, and that function is then run, and we assert it throws.
  2. I don't think that repetition is necessary.

@loopingz
Copy link

loopingz commented Mar 7, 2020

Not having a way to redirect the json output to a file can be an issue when launched within a test suite:
nyc mocha --recursive --exit --timeout=10000 -r ./node_modules/ts-node/register src/**/*.spec.ts -R json
As it will collide with nyc output and others. We could argue that we could do two runs: one for coverage and one for unit test result but it is overkilled no?
Am I missing something obvious?

@bryansray
Copy link

@boneskull Is it possible to remove the "extra" stuff that comes out when piping? For example, when I run something like:

npx cypress run --headless --reporter json > results.json

My results.json file comes out like this:


====================================================================================================

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:    3.8.3                                                                              │
  │ Browser:    Electron 78 (headless)                                                             │
  │ Specs:      1 found (test.spec.js)                                                             │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


────────────────────────────────────────────────────────────────────────────────────────────────────
                                                                                                    
  Running:  test.spec.js                                                                    (1 of 1)
{
  "stats": {
    "suites": 1,
    "tests": 12,
    "passes": 11,
    "pending": 0,
    "failures": 1,
    "start": "2020-05-06T02:31:09.690Z",
    "end": "2020-05-06T02:31:14.497Z",
    "duration": 4807
  },
 ..... removed for brevity ...

Obviously this isn't a valid JSON file because of the initial output. Is it possible to get rid of that and just pipe the JSON result?

@boneskull
Copy link
Contributor

that output is from cypress not mocha

@bryansray
Copy link

Oh, shoot. My apologies!

@boneskull
Copy link
Contributor

regarding nyc output, you could suppress that instead and write to file. not ideal...

this being said I would support adding the ability to swap the output stream (generally just process.stdout) in mocha, but it should be generic and applicable to all reporters. someone would need to send that PR though

this would allow these sorts of situations—mocha combined with other tools—to work together more nicely.

implementation would just be something like a “default” stream for each reporter—probably in the Base reporter—which can be swapped out via reporter options.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants